Coordinated Sampling in Communication Constrained Sensor Networks using Markov Decision Processes

The paper describes a Markov Decision Process (MDP) framework for coordinated sensing and adaptive communication in embedded sensor networks. The technique enables distributed sensor nodes to adapt their sampling rates in response to changing event criticality and the availability of resources (energy) at each node. The relationship between energy consumption, sampling rates, and utility of coordinated measurements is formulated as a stochastic model. The resulting model is solved as an MDP to generate a globally optimal policy that specifies the sampling rates for each node for all possible states of the system. This policy is computed offline before deployment and only the resulting policy is stored within each node. The on-line computational cost of executing the policy is minimal since it involves only a lookup of the optimal policy table. The optimal policy is computed under the assumption that the state of all sensors is completely observable. During execution, each sensor maintains a local estimate of the other sensor’s states. Sensors exchange their true local states when an information theoretic model of the uncertainty in the local state estimates exceeds a pre-defined threshold. Thus, the communication cost of executing a global policy is incurred only when a relatively large gain in the accuracy of the global state estimate is expected. We show results on simulated data that demonstrate the efficacy of this distributed control framework, the effect that the various model parameters have on the generated control policy, and compare the performance of the proposed controller with other policies.